Web Based Collection Selection Using Singular Value Decomposition

As the number of electronic data collections available on the internet increases, so does the difficulty of finding the right collection for a given query. Often the first time user will be overwhelmed by the array of options available, and will waste time hunting through pages of collection names, followed by time reading results pages after doing an adhoc search. Automatic collection selection methods try to solve this problem by suggesting the best subset of collections to search based on a query. This is of importance to fields containing large number of electronic collections which undergo frequent change, and collections that cannot be fully indexed using traditional methods such as spiders. This paper presents a solution to this problems of selecting the best collections and reducing the number of collections needing to be searched. Preliminary tests of the system, conducted on Web search engines, suggest that this will solve much of the Web based Collection selection problem.